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ABSTRACT 



The purpose of this study was to develop tests items 
with a minimum reading demand for Use with pupils at grade levels two 
through six. An item was judged to be acceptable if the item 
satisfied at least four of six criteria. Approximately 250 students 
in grades 2-6 participated in the study. Half of the students were 
given instruction to develop the ability to use concepts from the 
particle nature of matter. The test items developed were designed to 
assess this ability. Test items pertaining to concepts within a unit 
were administered to each of the instructed classes on the day 
following completion of instruction on the last concept. The same 
test items were administered to each of the noninstructed classes a 
day or so later. Although only two items met all six criteria, 104 
items met at least four of the six. Items related to each of the 25 
concepts that met the established requirements were developed, (BR) 
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THE PROBX.HI 



Introduction 

Concept development is presently a popular goal 
instruction at all levels. Knowledge of and ability to use 
concepts has become one of the essential outcomes of science 



in sclf,*.pce 



SC isiiff • 



instruction 



at the elementary school Icv-el. The determination of how vjcll elementary 
school pupils learn these concepts in and of science is consequem iy an 
essential part of good science teaching. These techniques have nut been 
widely available up to the present. "Quite candidly, it must be .-ulmitted 
that our present procedures of evaluation are still in the trial .-uid 
error stage (Lee, 1967, p,69), "The improvement of evaluative tools 
and techniques in elementary science instruction is one of the major 
priorities of science education" (Kuslan & Stone, 1968, p. 228). 



Statement of the Problem 

The purpose of this study was to develop test itemswich a 
minimum reading demand related to selected concepts from the schemn, the 
particle nature o.f matter, for use with pupils at grade levels two through 
six. An item was judged to be acceptable if the item satisfied al least 
four of the follo.^ing criteria; 



1. The proportion of the instructed group choosing tlu.- accepted 
response to each item, vzhen admin.istered as part of a posttest, i;j greater 
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than 0.50, the level attributable to random guessing. 

2. The pi'oportion of the instructed group choosing the accepted 
response to each item, v?hen administered as part of a posttest, is greater 
than when administered as part of a pretest. 

3. The proportion of the instructed group choosing the accepted 
response to each item, when administered as part of a posttest, is greater 
than the proportion of the nonins tructed group choosing the accepted response 
to the same item when administered as part of a posttest. 

4. The proportion of each instructed class choosing the accepted 
response to each item, when administered as part of a posttest, increases 
progressively vxith grade level, 

5. The item vjhen administered as part of a posttest to the 

instructed group as of nonextreme difficulty. ( ~ 2 sT”^X 5 q ^ -2 <r ) 

6. .The item v/hen administered as part of a posttest to the 

instructed group is a positive discriminator. ^ +0,30) 

Criteria for Test Items 

The constructs underlying the first four criteria are self- 
evident. The last two criteria are ibased on the fact that both the difficulty 
and discrimination of test items affect their usefulness. An item so easy 
that all answer it correctly or so difficult that none answer it correctly 
yields no information about relative levels of achievement. Items of low 
discrimination are not useful in differentiating between those students 
who possess the tested attribute and those who do not. The difficulty and 
discrimination values used in this study were X^q and ^ respectively, 

PROCEDURE 

Definition of Educational Objectives 

The objective of the instruction related to this study v/as to aid 
students in developing the ability to use selected concepts from the scheme, 

the particle nature of matter, in explaining natural phenomena. The test 
items developed in this study were designed to assess this ability with 
students at grade levels two through six. 
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An assumption underlying this study vras that tha prepared 
instructional sequence, as presented by the regular classroom teachei 



is effective in increasing the ability of students at grade levels tv70 
through si>: to use the selected concepts in explaining natural phenomena. 

Construction of the Test Items 

The item used in this study v?as of the alternate response 

pictorial item type presented via motion picture film. Alternate response 
items were chosen because they were adequate for the purpose and feasible 
for the motion picture medium. Pictorial items V7cre chosen so as to 
(1) reduce the demands on the reading and verbal ability of the students, 
and (2) assess the ability to use the concepts rather than reading or 
verbal ability. As motion was a relevant factor in some items, motion 
picture film was used to present the questions and model choices. 

Five separate test items V7ere developed for each of the 25 



concepts chosen for the study. 

If only one item per objective is used, the test will 
generally be less reliable than if more items are used. 

On the other hand, it may be assumed that, if a pupil 
can perform well on five or 10 items of similar content, 
he can perform well on 20 or 50 such items (Lindvall &■ Cox^ 

1969, p. 175). 

Because testing using motion pictures was unfamiliar to most 
of the students, extensive directions were prepared on ho\7 to mark the test 

booklet. 

Selection of the Population 

The criteria for population selection was that the school have 
two or more classes per grade and that the students had been randomly 
assigned to their class. The first criterion was established because the 
design of the study included an instructed and noninstructed class at each 

grade level. 

The second criterion was used to insure that the classes would be 
as similar as possible. Student date, such as IQ and scores on standardized 
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tests were obtained froin the school files to be used in ; verify in-- 



the equality of the instructed and noninstructed classes. The IQs 
were obtained from the Kuhlman-Anderson Test and the level of academic 
achievement by use of the Stanford Achievement Test (SAT). The 
Mathematical Coiiceijt, Paragraph Meaning, and science subtests of the 

SAT were chosen as being representative of abilities in mathematics, 
reading, and science. 

A Wisconsin school was selected to participate in the study 



as it satisfied the above criteria and the faculty was willing to participate 
in the program. The principal selected the teachers who v/ould be in the 
instructed and noninstructed classes. 

Collection of Data 

Instruction on the concepts was initiated with the instructed 
classes after all the test items had been administered to all classes 
(both instructed and noninstructed). Each concept was the subject of a 
30 minute lesson for one or two days of instruction by the regular 



classroom teachers. All of the instructed classes were taught the same 
lesson on the same day. 

On the day follovi/ing completion of instruction on the last 
concept in a given unit, test items pertaining to concepts within that 
unit were administered to each of the instructed classes. The same test 
items were administered to each of the noninstructed classes a day or so 
later. The test items were administered by the researcher to each class 
in its regular classroom. The verbal part of each question was read 
aloud by the researcher as the students viewed the written questions 
projected on the screen and printed on the test booklet. 
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RESULTS 



Tritroauct: 



ion 



The results of this study are presented in the following order: 
(1) selected concepts, (2) population, (3) item results for a representative 
concept, and (4) questions and model choices for the concept in (3). 

Selected Concepts 

The concepts from the scheme, the particle nature of m.atter, 
selected for this study are listed below. The order indicated was the order 
in vjhich the concepts were taught. 

Unit I 

1. All matter is made up of particles. 

2. Particles of matter have spaces between them. 

3. Particles of matter are very small. 

4. Particles of matter are in motion. 

5. Particles of matter move faster when the matter is heated. 

6. Particles of. matter move farther apart when the matter is heated. 

Unit II 

7. In the solid state, the particles of matter are packed together in a 
pattern and move within a small space. 

8. In the liquid state, the particles of matter are loosely clustered 
together and move about. 

9. In the gas state, the particles of matter are far apart and move freely. 

10. The state of matter can be changed from solid to liquid and from liquid 
to solid. 

11. The state of matter can be changed from liquid to gas and from gas to 
liquid. 



Unit III 

12. The push against the surface by a gas depends upon the number and rate 
of motion of the particles of the gas. 

13. Some particles of matter (molecules) are made up of simpler particles 

(atoms) . 

14. Some molecules are made up of only one kind of atom (element). 

15. Some molecules are made up of two or more kinds of atoms (compounds). 

16. Some samples of matter contain more than one kind of molecule (mixtures). 
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Unit IV 



17. 

18. 

19. 

20 . 
21 . 



&ich type of molecule is forinod from definite numbers and kinds of atoms. 
Particles of matter have, mass and occupy space. 

The average size and mass of the atoms of each element do not vary. 

Atoms are made up of particles; protons, electrons, and neutrons. 

Electric charges are associated v?ith the particles of matter. 



Unit V 



22. The particles of matter attract each other. 

23. The mass of an atom is determined the number and kind of particles 
that it contains. 

24. All atoms of a given element are made up of the same number of electrons 
and protons o 

25. Molecules are made up of atoms that are held together by electrical forces 

i 

Population 



The numbers of students in the instructed and iionins true ted classes 
completing all the test items, when admi.nistcred as part of a pretest and as 
part of a posttest, are given in Table 1, 

TABLE 1 

NUMBERS OF STUDEI\-TS IN THE INSTRUCTED AND NON INSTRUCTED 
CLASSES COMPLETING THE TEST ITEMS BY GRiVDE LEVEL 





Instructed 


Non ins t r u c t ed 


Grade Level 


Pre 


Post 


Pre 


Post 


2 


26 


26 


23 


23 


3 


23 


22 


24 


24 


4 


27 


26 


27 


27 


5 


26 


25 


27 


26 


6 


26 


25 


25 


2k 




128 


124 


126 


124 



The instructed and nonins true ted classes at a given grade level 
were assumed to be comparable because the students were assigned to classes 
irrespective of their academic ability. 7'he credibility of this assumption 
was testable within the second and fifth grades as current standardized 
test data were available for these grades. The assumption was reasonable 
as none of the pairs of classes were significantly different (per t-tests, 

= 0,01) with respect to IQ and mathematical, reading and science 
abilities as measured by the subtests of the Stanford Achievement Test. 
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iCeifl Results 

Due to space limitations, one concept (No. 11) was chosen as an 
example. The results related to the items that tested Concept No. 11 are 
shown in Tables 2, 3, and 4. Table 2 includes the proportion of each of the 
classes choosing the accepted response to each item when administered as 
part of a pretest and as part of a posttest. Presented in Table 3 are the 
item statistics (X 3 Q and ^ ) for each item when it was administered as 
part of a posttest to the instructed group. Table 4 indicates V 7 hich items 
satisfied the several item criteria. It is noted from Table 4 that each 
item related to Concept No. 11 satisfied five criteria ■ and failed to meet 
Criterion #4. 

Questions and Model Choj.ces 

The questions and model choices for the items related to 
Concept No. 11 are illustrated on Page 9. The models for items II-ll, 

II-18, and 11-20 are dynamic; the particles move and change their 
configurations (i.e. particle configurations change from a gas configura- 
tion to a liquid configuration), 

SUMMA.RY 

Examination of the results of the dataj from the frame of 
reference of "which items meet four of the six criteria" and hence make up a 
pool of acceptable items, reveals that although only two items meet all six 
criteria, there are 104 items that meet at least four of the six (Table 5). 
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Indicates the criterion v/as satisfied 
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Questions and Model Choices for Concept No. 11 



II-3 

VIhicb model i.s more useful 
in explaining boiling? 



A 


B 


■ 

; o 
o 

oOoQ;)| 


o o o 
o 

o o 







II - 11 

Which model is more useful 
in explaining a gas changing 
to a liquid? 



II- 9 



Which model is more useful in 
explaining a liquid changing 
to a gas? 




V)hich model is more useful in 
explaining vjhat happens after a 
liquid is heated until it becomes 
a gas? 



ERIC 



A 






O 



o 



o 



■o 

O O 

o 

liq -> gas 



B 



T( ///!///( /f 



o o 
0O o 

o O 



gas-> liq 



II- 20 



A 




sol-» liq 



B 



f ////// rrr/. 


0 ^ 
o 


'0 


0 ( 




0 0 




0 


o 



liq gas 



Which model is more useful 
in explaining what happens 
when a gas is cooled until 
it becomes a liquid? 



B 



mUlIf / 
gas liq 



\niII7ZU 

O OO 

o§o 
Oo o 

o ° 

o o 



liq gas 
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TABLK 5 



FREQUKMCY DISIRIBl’IION OF ITEiS ACCORDING 
TO TOTAL NUMBER OF CFilTERIA SATISFIED 



Total Number of 
Criteria Satisfied 



Number of Items 



6 

5 

4 

3 

2 

1 

0 



2 

71 

31 

17 

4 

0 

0 



The one criterion not met by most items was #4 (Table 6) , It 

! 

seems that the items either do not differentiate between levels of under'- 
standing of the scheme, the particle nature of matter, or the instructional 
sequence has not extended beyond the levels possible for children in the 
lower grades. 

TABLE 6 

. FREQUENCY DISTRIBUTION OF ITEMS THAT SATISFIED EiVCH 
CRITERIA ARRANGED ACCORDING TO CRITERION NUMBER 



Criterion 

Number 

1 

2 

3 

4 

5 

6 



Number 
of Items 

108 

104 

106 

2 

111 

119 



It is noted from Table 7 that the mode of the item, X^q distribution is in 
the range -1,00^ to - 0 , 50 ^ and the items are distributed in an approxi- 
mately "normal" manner. A range of values is often stated as a desired 
goal for the items within a test or an item pool, when one wishes to 
maximally discriminate among all students. 



n 



TABLE 7 



FREQUENCY DISTRIBUTION OF ITE-IS 


; ACCORDING 


TO X R 
50 


Range of (<r units) 


Number 


of Items 


-oo 2.00 


8 




-2.00 1.50 


16 




-1.50 1.00 


22 




-1.00 0.50 


33 




-0.50 — -0.00 


22 




0.00 — +0.50 


12 




+0.50 FI. 00 


4 




+1.00 — +1.50 


2 




+1.50 — +2.00 


0 




+2.00 + oo 


0 





It is noted from Table 8 that that the frequency distribution of 
items according to the range of^va3.ues, an index of item discriminating 
power, is positively skewed and its mode is the range of +1.00 to +1.25., 

The high p values indicate that the items are effective in discriminating 
between the students v-7ho possess the tested ability and those v/ho do not. 

TABLE 8 

FREQUENCY DISTRIBUTION OF ITEMS ACCORDING TO RANGE 



Range of /3 



Number of Items 



2.50 — 


oo 


5 


2.25 — 


2.50 


3 


2.00 — 


2.25 


5 


1.75 — 


2.00 


1 


1.50 — 


1.75 


16 


1.25 — 


1.50 


20 


1.00 — 


1.25 


29 


0.75 — 


1.00 


25 


0.50 — 


0.75 


14 


0.25 — 


0.50 


1 


0.00 — 


0.25 


0 



ERIC 



12 



• When usin^ the itciu ]fOol developed here, care must be eKcrcizod 
to select itcuns according to concept if the item is to determD.ne levels of 
achievement v/ithin the scheme, the particle nature of matter. A total of 
104 acceptable items (those that satisfied four at the six criteri.a) were 
developed in this study. Items related to each of the 25 concepts that meet the 
estctblished requirements were developed and in most cases more than one item 
is available. It is recoird?. ended that more than one item he selected for each 
concept to increase the precision of measurement 
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